A New Vision of Collaborative Active Learning

Active learning (AL) is a paradigm where an active learner has to train a model (e.g., a classifier) which is in principle trained in a supervised way. In contrast to supervised learning, AL has to be done by means of a data set where a low fraction of samples is labeled or even with an initially unlabeled set of samples. To obtain labels for the unlabeled samples, the active learner has to ask an oracle (e.g., a human expert) for labels. In most cases, the goal is to maximize some metric or task performance and to minimize the number of queries at the same time. In this article, we first briefly discuss the state-of-the-art and own, preliminary work in the field of AL. Then, we propose the concept of collaborative active learning (CAL). With CAL, we will overcome some of the harsh limitations of current AL. In particular, we envision scenarios where the expert or the gold standard may be wrong for various reasons. There also might be several or even many experts with different expertise, the experts may label not only samples but also supply knowledge at a higher level such as rules, and we consider that the labeling costs depend on many conditions. Moreover, in a CAL process human experts may even profit by improving their own knowledge, too.